Introduction
Agricultural and natural resource leadership development programs have been in existence since the mid-1950’s based on an initial grant from the Kellogg foundation (Strickland, 2008). The purpose of many such leadership development programs is to develop opinion leaders within the agricultural and natural resource community (Lamm, Lamm, & Strickland, 2013). Opinion leadership is defined as the two-step communication process identified by Lazarsfeld, Berelson, and Gaudet (1948) whereby information is disseminated to opinion leaders and then processed and diffused out to the opinion leader’s social and professional networks. With over 30 programs around the globe, and more than 9,000 program alumni this confederation of programs represents a significant conduit to the best and brightest within the agricultural industry (Strickland, 2008; Kelsey, 2003). However, there has been a lack of participation from program alumni in the international alumni program, the International Leadership Alumni Conference.
Alumni programs have been found to be extremely valuable to the organizations they are affiliated with. Yet, based on their review of civic leadership programs for adults in California Azzam and Riggio (2003) noted “The involvement level of alumni was a major concern facing many leadership programs” (p. 64). Alumni that participate in formal alumni programs provide financial support as well as less tangible sponsorship of the original organization (Newman & Petrosko, 2011). When leadership development program participation ends after a class graduates, the program, as well as the broader agricultural and natural resource industry, loses a potentially valuable asset (Kelsey, 2003). Research has found that one of the main predictors for participation in an alumni program is the experience an individual had with the underlying program. For example, Newman and Petrosko (2011) found when individuals had a better experience they were less likely to participate.
This study addressed the problem of lack of leadership development program participation in alumni programs by analyzing the issue from a program satisfaction and opinion leadership perspective, specifically in non-higher education contexts (Gill, 1998). The problem was analyzed from a multi-level perspective taking into consideration not only individual level of satisfaction with their leadership development program experience but also the group mean level of satisfaction for all individuals within a particular leadership development program class. The importance of group level satisfaction has been found to be relevant to numerous outcomes including group performance (Robbins & Judge, 2009) and group goal achievement (Springer, Stanne, & Donovan, 1999). The role of opinion leadership in moderating the relationship between satisfaction and intention to participate will be considered. To improve the effectiveness of leadership development program alumni programs, program directors may find understanding the role that individual satisfaction, group level satisfaction, and opinion leadership, play to be informative. The results of the study will provide leadership educators a starting point to “encourage open dialogue and collaborative research efforts that develop methods of monitoring and evaluation to create formative, summative, and developmental processes to establish a standard of quality for Leadership Education programs” (Andenoro et al., 2013, p. 10).
Conceptual Framework
The present study is grounded in Richey’s (1992) systemic model of factors predicting employee training outcomes. Within the model organizational perceptions and learner characteristics are included as factors related to and predictive of outcomes. As a conceptual framework, the model undergirds the functional aspects of the current study. Specifically, acknowledging learner perceptions of the program as well as learner characteristics to predict an outcome, in this case participating in an alumni program. Functionally the study relies on previously established findings and empirically observed relationships between individuals and alumni program participation.
Organizational perceptions. As a measure of organizational perceptions predicting outcomes (Richey, 1992), previous research has examined the relationship between alumni program participation and program satisfaction. Within the literature, alumni associations within higher education contexts have been examined more thoroughly than other potential alumni associations such as those affiliated with adult leadership development programs. For example, Newman and Petrosko (2011) found that there were a number of criteria that predicted participation in university alumni associations when analyzed using logistic regression. One of the findings was that positive feelings (satisfaction) about student experiences negatively predicated alumni participation. This finding was inconsistent relative to other literature that found individual satisfaction was related to higher performance (Netemeyer, Maxham, & Lichtenstein, 2010) and that performance outcomes and team level satisfaction were related (Li, Li, & Wang, 2009). However, these results may be similar to those of Iaffaldano and Muchinsky (1985) who found in their meta-analysis that the relationship between satisfaction and performance is frequently overstated due to measurement error and small sample sizes. The inconsistency in findings provides for opportunities to examine satisfaction at the individual and group levels related to other intended outcomes, such as intentions. Additionally, the role of moderators in describing the nature of relationships between satisfaction and outcomes is also relevant (Judge, Thorensen, Bono, & Patton, 2001).
Learner characteristics. In addition to organizational perceptions, individual learner characteristics are also acknowledged within the systemic model of factors predicting employee training outcomes (Richey, 1992). Whether explicitly stated or implied, a consistent characteristic of many agricultural and natural resource leadership development programs is the cultivation and development of opinion leadership (Lamm et al., 2013). The intent is to develop the confidence and capacity of individuals to first been seen as leaders and a source of knowledge and insights within their circle of influence and to then step into this role and provide both knowledge and insights (Lazarsfeld et al., 1948). Opinion leadership has been found to be valuable within agricultural and natural resource contexts based on the increased credibility of information shared via opinion leaders (Kelsey, 2003; Rogers, 2003).
As it relates to alumni program participation, opinion leadership was deemed an appropriate measure based on the nature of the concept. In particular, opinion leaders, by definition are expected to be consumers of information and data sources, therefore, ongoing development programs, such as alumni programs, and opportunities to increase information exposure through ongoing programs are anticipated (Whent & Leising, 1992). According to Azzam and Riggio (2003) and their review of community based civic leadership programs in California a descriptive analysis of both program characteristics and alumni programming were examined. The researchers found, “regardless of the number of alumni, many programs are faced with the challenge of keeping their alumni connected” (p. 63). Empirically, as it relates to intentions to participate in alumni organizations, Newman and Petrosko (2011) found that involvement with their institutions positively predicted participation in university alumni associations.
Purpose & Research Objectives
The purpose of this study was to examine how agricultural and natural resource leadership development program alumni intentions to participate in an alumni program are influenced by their satisfaction with their program experience and current level of opinion leadership. The study was driven by the following research objectives:
- Determine whether there are group level differences between intentions to participate in an alumni program.
- Determine whether individual and group level satisfaction is related to intentions to participate in an alumni program.
- Determine whether individual and group level opinion leadership is related to intentions to participate in an alumni program.
Methods
To address the research purpose and objectives, the study employed a descriptive and causal-comparative research design. The causal-comparative method was employed according to the expectation that a cause, individual and group level program satisfaction and/or opinion leadership, would be associated with differences in effects, in particular, intention to participate in an alumni program (Kirk, 1995; Lamm, Sapp, & Lamm, 2016). The population for this study was alumni of agricultural and natural resources leadership development programs. A census of 23 programs across the United States was conducted. A census, or comprehensive sample, frequently yields the most comprehensive data within a population of interest (e.g. Ary, Jacobs, Sorensen, & Razavieh, 2010; Rossi, Lipsey, & Freeman, 2004).
Data were collected in the spring of 2014 using an online survey tool. A total of 6,783 surveys were sent via email, with 771 were returned based on inaccurate email addresses, for a net of 5,790 potential respondents. There were 2,505 questionnaires completed for a total response rate of 43%; however, 304 responses were less than 10% complete and were removed, for an effective response rate of 38%. The response rate was deemed acceptable given existing social science thresholds (Baruch & Holtom, 2008). Responses were returned from 262 classes from the 23 programs. Missing data were coded as -999 and noted in the software analysis syntax (Muthén & Muthén, 2012).
Demographic data were obtained through respondent self-report. The sample was 67.4% male and 32.6% female. The average age of respondents was 51 (M = 51.1, SD = 11.3) with a range of ages between 23 and 90. From an ethnicity perspective less than 1% (n = 30, coded as 1) of respondents identified themselves as Hispanic/Latino(a)/Chicano(a). In regard to respondents’ race individuals could self-identify in more than one category: 94.3% (n = 2075) identified themselves as White, less than 1% of identified themselves as Asian (n = 182) American Indian (n = 33), African American (n = 28), or Other (n = 55).
Organizational perception variable – program satisfaction. Leadership development program satisfaction was collected using a measure developed by Judge, Boudreau, and Bretz (1994). The original measure was found to have a Cronbach’s a of .85 within a work context and has been found to be reliable in educational settings as well (e.g. Lamm, Carter, & Melendez, 2014). The measure includes three items. The first item asked respondents to indicate if they were satisfied with their experience by responding yes (coded as a 1) or no (coded as a 0). Based on findings within the literature, binary questions can reduce the cognitive load on respondents without diminishing the validity of findings (e.g. Martín, Román, & Gonzaga, 2018). The second item asked respondents to indicate how they felt about their experience in general using a five-place circular face satisfaction series originally developed by Kunin (1955). The five items were coded from 1 = least satisfied to 5 = most satisfied. Finally, participants reported the percentage of time that they were satisfied with their experience; available responses ranged from 0% to 100%. A satisfaction index score was calculated by multiplying each of the three items acknowledging the influence the binary item within the analysis. Calculated index values ranged from 0 to 5.
Individual characteristic variable – opinion leadership. Opinion leadership was measured using a scale developed by Childers (1986). Participants responded to six questions each using a five-place bipolar response. A one indicated the negative statement and a five indicated a positive statement. Reliability was calculated with a Cronbach’s a= .90.
Outcome variable – intention to participate in an alumni program. The variable of interest, intention to participate in an alumni program, was measured using a three-item researcher developed scale. The scale was developed using Ajzen’s (2002) recommendations for developing behavioral intention measures within a Theory of Planned Behavior (Ajzen, 1991) context. Reliability was calculated with a Cronbach’s a= .91.
Analysis
Data distributions were examined using descriptive statistics. Two-level multi-level models were used to analyze intention to participate in an alumni program to account for the nested data structure (individuals within groups) (Snijders & Bosker, 2012). The first level measured the differences between individuals within groups, and the second level measured the differences between groups. Mplus version 7 (Muthén & Muthén, 2012), software was used for this analysis. There were a total of four models analyzed. The first model was an unconditional model used to calculate the intraclass correlation coefficient. The second model examined how intention varied when group level satisfaction was grand mean centered and accounted for. The third model examined if program satisfaction and opinion leadership were predictors of intention to participate in an alumni program across groups. The final model examined if group satisfaction was a predictor of intention to participate in an alumni program when controlling for both group mean centered individual satisfaction and opinion leadership.
Results
Results from the study were analyzed descriptively to establish mean and standard deviation values for the variables of interest. In particular intention to participate in an alumni program was treated as the dependent variable. Satisfaction with an individual’s particular leadership development program and level of opinion leadership were treated as the independent variables. Related to satisfaction with their leadership development program, within a range of 0 to 5, respondents had a mean score of 3.89. The results of the descriptive analysis are presented in Table 1.
Table 1
Descriptive statistics (n = 2,200)
Correlation analysis between intention to participate in an alumni program, individual program satisfaction, and opinion leadership were completed. Intention to participate in an alumni program had a positive correlation with program satisfaction and a negative correlation with level of opinion leadership. Correlation coefficients and statistical significance between variables are provided in Table 2.
Table 2
Intercorrelations Between Intention, Satisfaction, and Opinion Leadership (n = 2,200)
Four multi-level models were analyzed including individual, group level, and interaction effects. Based on the analysis completed, model one resulted in an average cluster size of 8.4. In the present research study cluster size refers to the average class size within the analysis. Additionally, model one was used to test for the effect size attributable to group membership and to determine whether further multi-level model analysis was warranted. From the analysis an intraclass correlation coefficient of .08 was obtained. According to LeBreton and Senter (2008) an intraclass correlation coefficient “value of .01 might be considered a ‘small’ effect, a value of .10 might be considered a ‘medium’ effect” (p. 838). Based on guidance in the literature the calculated intraclass correlation coefficient for this study was deemed appropriate to warrant further investigation of the multi-level nature of the data (LeBreton & Senter, 2008; Snijders & Bosker, 2012).
The second model analysis introduced the group satisfaction predictor. Based on the analysis output from the Mplus software (Muthén & Muthén, 2012) the estimated proportion of variance between groups explained by the second model was approximately 0.2%.
In the third model individual satisfaction and opinion leadership were group mean centered and analyzed. Group mean centering was completed to avoid confounding in the model (Snijders & Bosker, 2012). Based on the analysis output from the Mplus software (Muthén & Muthén, 2012) the two level-one variables accounted for approximately 3.2% of the level-one variance. Further analyses were undertaken to determine if model fit was improved by removing one of the predictors, in both cases model fit was worse, significantly so using chi-square tests, so the two level-one variables were retained. Specifically, an individual whose satisfaction was equal to the mean satisfaction score of their group, a one-unit increase in individual satisfaction score resulted in a .05 unit increase in intention to participate in an alumni program. However, for an individual whose opinion leadership score was equal to the mean opinion leadership score of their group, a one-unit increase in individual opinion leadership resulted in a .11 unit decrease in intention to participate in an alumni program.
The fourth and final model again group mean centered individual satisfaction and opinion leadership at level one to safeguard against level two interaction problems. Based on the fourth model, for groups a one unit increase in satisfaction resulted in a .08 unit increase in intention to participate in an alumni program. No other significant effects were found. Additionally, no moderation effects of opinion leadership were observed. Model effects and statistical significance for all four models are presented in Table 3.
Table 3
Effects for Multi-Level Models (n = 2,200, groups = 262)
* p < .05 (one-tailed), ** p < .05 (two-tailed), *** p < .001
Conclusions, Implications, and Recommendations
The results of the present research provide a basis for both leadership development program application as well as methodological consideration. From a leadership development program application perspective, the results provide a quantitative view into the effect that both individual and group level variables can have and intended outcomes. This is noteworthy as there has been very little empirical research into the effect that cohort or class experience can have on learner outcomes as it relates to adult leadership development programs. Azzam and Riggio (2003), specifically recommended, “an evaluation of alumni across various leadership programs may also provide valuable insights into how leadership programs affect alumni and what are the best ways to keep alumni connected” (p 66). The results of the study indicate that although the effects were not large from a practical perspective, they were statistically significant.
Prior to further discussing the results and drawing conclusions and recommendations it is important to note a number of limitations associated with the study. First, the study employed a census approach; however, with a 38% response rate there were a large number of potential respondents that are not represented in the data. Although checks were made to ensure results were not biased according to recommendations in the literature for accounting for non-response (Lindner, Murphy, & Briers, 2001), there is the potential that the results may not be representative.
An additional limitation is the relatively small effect sizes that were observed from the analysis. It is important that the results be considered in the manner in which the data were collected, in particular, the results are not intended to be generalizable per se, but rather an empirical measure at a particular point in time with a particular set of programs. Subsequent implications and recommendations should be considered within this context. A general recommendation would be to replicate the findings from this study with similar adult leadership development program classes to better illuminate to nature of the relationships between the variables.
From a program application perspective, understanding the context and needs of adult learners is well established in the literature (e.g. Merriam, Caffarella, & Baumgartner, 2007), the results of the present study also indicate the importance of considering the group mean experience as well. Therefore, a recommendation for leadership educators, particularly those working with adult leadership development programs, is to be mindful of the class or cohort mean experience while also balancing the needs of individual learners.
Specific to the results of the present study the two variables of interest, group level satisfaction and group level opinion leadership, were observed to interact with an individual’s intention to participate in an alumni program in a noteworthy manner. First, satisfaction was observed to have a positive relationship with intention to participate in alumni programs at the group level. This finding was inconsistent with that of Newman and Petrosko (2011), but consistent with the findings of Netemeyer, Maxham, and Lichtenstein (2010). An implication of this finding is that it is possible for the overall group mean satisfaction to have an effect on intention to participate in alumni programs. Therefore, a recommendation would be for leadership educators to consider not only individual satisfaction needs, but also to balance the needs of individuals against the needs of the entire class.
More unexpectedly were the results of the opinion leadership analysis. Specifically, opinion leadership was expected to be positively related to intention to participate in alumni programs (Newman & Petrosko, 2011); however, the observed results indicated the opposite, that opinion leadership was negatively related to intention to participate in alumni programs. An implication from this result may be that classes that have higher levels of mean opinion leadership may have an insular effect whereby individuals may not see the need or value in participating in alumni programs based on their perceived performance as reinforced by their cohort. Perhaps strong opinion leaders do not see the need to participate in ongoing activities because they have already achieved a level of ability that they are content with. These findings are consistent with previous research that found that leadership characteristics were related to engagement (Lamm, Lamm, Rodriguez, & Owens, 2016).
A recommendation would be for leadership educators to be aware of this potential and to consider recruiting individuals with a diverse amount of opinion leadership, in particular those individuals that have the most potential to benefit from the program based on lower levels of initial opinion leadership. A second recommendation would be for leadership educators to periodically remind groups that embracing a positive attitude towards a behavior, such as continuous learning, has been shown to be predictive of behavioral intent (Ajzen, 1991). Therefore, regardless of perceived level of opinion leadership within the group, individuals may be more inclined to participate in alumni programs.
A second outcome from the present study, in addition to the leadership development program applications, is the methodological approach employed. Despite the similarities amongst agricultural adult leadership development programs (e.g. Kaufman, Rateau, Carter, & Strickland, 2012), there has been little research examining the group level effects on learner outcomes. The results of the present study indicate the use of multi-level models as a potential methodological approach that will help to provide group-level insights. A recommendation for future research is to consider both multi-level model analysis techniques as well as multi-level structural equation model (SEM) analysis. The use of these statistical analysis approaches has the potential to further identify relationships at the group-level that may be related to learner or program outcomes.
References
Ajzen, I. (1991). The theory of planned behavior. Organizational Behavior and Human Decision Processes, 50(2), 179-211. doi: 10.1016/0749-5978(91)90020-T
Ajzen, I. (2002). Constructing a TpB questionnaire: Conceptual and methodological considerations. Retrieved from http://www.people.umass.edu/aizen/pdf/tpb.measurement.pdf
Andenoro, A. C., Allen, S. J., Haber-Curran, P., Jenkins, D. M., Sowcik, M., Dugan, J. P., & Osteen, L. (2013). National Leadership Education research agenda 2013-2018: Providing strategic direction for the field of leadership education. Retrieved from Association of Leadership Educators website: http://leadershipeducators.org/ResearchAgenda.
Ary, D., Jacobs, L. C., Sorensen, C., & Razavieh, A. (2010). Introduction to research in education. Belmont, CA: Wadsworth Cengage Learning.
Azzam, T., & Riggio, R. E. (2003). Community Based Civic Leadership Programs: A Descriptive Investigation. Journal of Leadership & Organizational Studies, 10(1), 55–67. https://doi.org/10.1177/107179190301000105
Baruch, Y., & Holtom, B. C. (2008). Survey response rate levels and trends in organizational research. Human Relations, 61(8), 1139-1160. doi:10.1177/0018726708094863
Childers, T. L. (1986). Assessment of the psychometric properties of an opinion leadership scale. Journal of Marketing Research, 23(2), 184-188.
Gill, J. L. (1998). Alumni programs: Principles and practice. In W. W. Tromble (Ed.), Excellence in advancement: Applications for higher education and nonprofit organizations (pp. 241–280). Gaithersburg, MD: Aspen Publishers.
Iaffaldano, M. T., & Muchinsky, P. M. (1985). Job satisfaction and job performance: A meta-analysis. Psychological Bulletin, 97(2), 251-273. doi:10.1037/0033-2909.97.2.251
Judge, T. A., Boudreau, J. W., & Bretz, R. D. (1994). Job and life attitudes of male executives. Journal of Applied Psychology, 79(5), 767-782. doi:10.1037/0021-9010.79.5.767
Judge, T. A., Thoresen, C. J., Bono, J. E., & Patton, G. K. (2001). The job satisfaction–job performance relationship: A qualitative and quantitative review. Psychological Bulletin, 127(3), 376-407. doi:10.1037/0033-2909.127.3.376
Kaufman, E. K., Rateau, R. J., Carter, H. S., & Strickland, L. R. (2012). What’s context got to do with it? An exploration of leadership development programs for the agricultural community. Journal of Leadership Education, 11(1), 121-139.
Kelsey, K. (2003). Do agricultural leadership programs produce community leaders? A case study of the impact of an agricultural leadership program on participants’ community involvement. Journal of Agricultural Education, 44(4), 35-46. doi: 10.5032/jae.2003.04035
Kirk, R. E. (1995). Experimental design: Procedures for the behavioral sciences (3rd ed.). Pacific Grove, CA: Brooks/Cole.
Kunin, T. (1955). The construction of a new type of attitude measure. Personnel Psychology, 8, 65-78.
Lamm, K. W., Carter, H. S., & Melendez, M. W. (2014). Investigating the linkage between intrinsic motivation and project team satisfaction in undergraduate leadership students. Journal of Agricultural Education. 55(3), 1-14. doi: 10.5032/jae.2014.03103
Lamm, A. J., Lamm, K. W., Rodriguez, M. T. & Owens, C. T. (2016). Examining leadership style influence on engagement in a national change process: implications for higher education. Journal of Leadership Education. 15(4), 103-115. doi: 1012806/V15/I4/R1
Lamm, A. J., Lamm, K. W., & Strickland, L. R. (2013). Focusing on the future: understanding faculty intent to lead the land grant system. Journal of Agricultural Education. 54(4), 92-103. doi: 10.5032/jae.2013.04092
Lamm, K. W., Sapp, L. R., & Lamm, A. J. (2016). Leadership Programming: Exploring a Path to Faculty Engagement in Transformational Leadership. Journal of Agricultural Education, 57(1), 106-120. doi: 10.5032/jae.2016.01106
Lazarsfeld, P., Berelson, B., & Gaudet, H. (1948). The people’s choice (2nd ed.). New York: Columbia University Press.
LeBreton, J. M., & Senter, J. L. (2008). Answers to 20 questions about interrater reliability and interrater agreement. Organizational Research Methods, 11(4), 815–852.
Li, F., Li, Y., & Wang, E. (2009). Task characteristics and team performance: The mediating effect of team member satisfaction. Social Behavior and Personality, 37(10), 1373-1382. doi:10.2224/sbp.2009.37.10.1373
Lindner, J. R., Murphy, T. H., & Briers, G. E. (2001). Handling nonresponse in social science research. Journal of Agricultural Education, 42(4), 43-53. doi:10.5032/jae.2001.04043
Martín, J. C., Román, C., & Gonzaga, C. (2018). How different n-point likert scales affect the measurement of satisfaction in academic conferences. International Journal for Quality Research, 12(2), 421-440. doi: 10.18421/IJQR12.02-08
Merriam, S. B., Caffarella, R. S., & Baumgartner, L. M. (2007). Learning in adulthood: A comprehensive guide. (3rd ed.). San Francisco: Josey-Bass.
Muthén, B., & Muthén, L. (2012). Mplus version 7: User’s guide. Los Angeles, CA: Muthén & Muthén.
Netemeyer, R. G., Maxham III, J. G., & Lichtenstein, D. R. (2010). Store manager performance and satisfaction: Effects on store employee performance and satisfaction, store customer satisfaction, and store customer spending growth. Journal of Applied Psychology, 95(3), 530-545. doi:10.1037/a0017630
Newman, M. D., & Petrosko, J. M. (2011). Predictors of alumni association membership. Research in Higher Education, 52(7), 738-759. doi:10.1007/s11162-011-9213-8
Richey, R. (1992). Designing instruction for the adult learner: Systematic training theory and practice. London: Kogan Page.
Robbins, S. P., & Judge, T. (2009). Organizational behavior. Upper Saddle River, N.J: Pearson Prentice Hall.
Rogers, E. M. (2003). Diffusion of innovations (5th ed.). New York, NY: Free Press.
Rossi, P. H., Lipsey, M. W., & Freeman, H. E. (2004). Evaluation: A systematic approach. Thousand Oaks, CA: Sage.
Snijders, T., & Bosker, R. (2012). Multilevel Analysis. An introduction to basic and advanced multilevel modeling. 2nd edition. London. SAGE Publishers.
Springer, L., Stanne, M. E., & Donovan, S. S. (1999). Effects of small-group learning on undergraduates in science, mathematics, engineering, and technology: A meta-analysis. Review of Educational Research, 69(1), 21-51. doi:10.2307/1170643
Strickland, L. R. (2008). Relationship between emotional intelligence and leadership style of leaders in Florida agriculture (Master’s thesis). University of Florida, Gainesville, FL.
Whent, S., & Leising, J. (1992). A twenty-year evaluation of the California agricultural leadership program. Journal of Agricultural Education, 15(3), 32-39.