Evaluation of the National FFA Nominating Committee Training
Dr. Jacklyn Bruce
Assistant Professor of Youth and Adult Leadership Development Department of Agricultural and Extension Education Pennsylvania State University
University Park, PA 16802-1294
Dr. Kristina Ricketts
Assistant Professor of Youth and Adult Leadership Development Department of Agricultural and Extension Education Pennsylvania State University
University Park, PA 16802-1294
Abstract
The purpose of this qualitative study was to evaluate if the environment of the National FFA Organization’s Nominating Committee training fosters transfer of training to the job they are required to perform, selecting National FFA Officers. Nominating Committee members, advisors, National FFA staff, and National Officer candidates participated in the study. The research methods included semi- structured focus group interviews, document analysis, and a short survey. Major findings of the study noted that most areas of the training environment encourage transfer; however, some facets of the environment could be improved to promote training transfer. Recommendations include offering pre-convention training materials in a variety of formats, widening the opportunities for pre-convention meetings to accommodate schedules, continuing to allow Nominating Committee members to set their own cultural norms, and clarifying the roles and responsibilities of the adult consultants with respect to the selection process.
Introduction
Within the National FFA Organization, young people have the opportunity to serve in many ways. One of the most distinguished ways is through participation on the National FFA Nominating Committee (or Nom Com). These students interview and select the national officers at FFA National Convention. Policies and procedures have been put into place in an effort to ensure that the Committee
is qualified to select the national officers. Beyond that, training is provided to the individuals selected to the Committee, so they will be prepared for the rigorous task that they will undertake. The pre-convention training includes several conference calls between Nom Com members, the Adult Consultants (Nom Com advisors) and National FFA staff to discuss the selection process and answer any questions. Nom Com members also receive all preparatory/study materials that National FFA Officer Candidates receive in order to become familiar with resources. Finally, Nom Com also receives specialized study materials to help them understand the scoring process, behavioral interviewing, and the written exam. Once Nom Com members arrive at National FFA Convention an intense training begins in behavioral interviewing, learning to interpret non-verbal cues, scoring of applications, written essays and exams, and developing a sense of internal social context.
While this practice of selecting and training highly qualified individuals has been in place for some time, there has never been a formal evaluation done to assess the effectiveness of the Nom Com’s training in relation to the job function they perform.
Theoretical Framework
Leadership and the FFA Organization
The mission of the FFA Organization is to make a positive difference in the lives of its members by developing their potential for premier leadership, personal growth, and career success through agricultural education (National FFA Organization, 2001). For the FFA Organization, leadership means holding office, participating in career development events, public speaking, chapter meetings, and parliamentary procedure (Seevers & Dormody, 1994). In fact, in a tri-state study, Seevers and Dormody (1994) found that FFA members reported that holding an office was one of the activities most responsible for their personal leadership development.
The National FFA Nominating Committee is a group of individuals charged with task of selecting a slate of new National FFA Officers during the National FFA Convention. These individuals, all former State FFA Officers, are in part selected by the outgoing team of National Officers.
Training and Training Transfer
Training is beneficial for both participants and the larger organization. The organization benefits through improved performance and increased productivity. Trainees benefit by developing new skills and improving their performance (Elangovan & Karakowsky, 1999). Garavaglia (1993) defined transfer of training as the successful and long-term use of skills learned in training and on the job.
Noe (1986) defined motivation to transfer skills as the desire of the students to use the material they learned in training on their job. Motivation is further broken down into five elements: perceived relevance of training, choice in attending training, outcome expectancies, self-efficacy, and job involvement.
Noe’s (1986) ability related factors affect the capacity of learners to transfer training. Knowledge acquisition and situation identification are ability related factors. Environmental factors are features of a work environment that aides or hinders transfer of training.
Job related and organization related are Elangovan and Karakowsky’s (1999) categories of environmental factors. Job related factors are specific to a job setting. They vary from situation to situation; however, consistent elements are: job requirements and norms and group pressure. Organizational factors apply to the whole organization and are: rewards and organizational culture.
Organizations, including the National FFA Organization, spend a great deal of money on training. Training dollars could be maximized if organizations examined their training processes, using the factors identified by Elangovan and Karakowsky (1999) and Noe (1986). Ideally, members would utilize more of the concepts promoted in the training, thus becoming more effective in the roles assigned.
Purpose of the Study
Garavaglia (1993) advises us that by evaluating the transfer of training, one can demonstrate how training improves an organization (in this case the National FFA Nominating Committee) and verify training effectiveness. The purpose of this study was to evaluate, using the factors advocated by Elangovan and Karakowsky (1999) and Noe (1986), the National FFA Nominating Committee training as part of the larger National FFA Officer selection process. The researchers decided to
approach the evaluation from this perspective due to the strict nature of the training content.
Methodology
The National FFA Nominating Committee evaluation took place directly before, during and after the National FFA Convention in October 2006. Several methods were used to collect data for this evaluation. Sources of information included training documents, observations, focus groups, and surveys. Document analysis determined common themes among the training materials. Participant observation determined effects of the training environment on the training transfer to the responsibilities of the Committee. Observations were carried out at National FFA Convention during the National Officer selection process. Focus groups demonstrated the qualitative effects of the experience and the effectiveness of the training and training environment, and were conducted during and after National FFA Convention. Finally, Nominating Committee members, advisors and National Officer Candidates were asked to complete a survey on their perceptions of the Nominating Committee process and National Officer selection process.
These results were used to triangulate focus group findings.
Focus groups are used to acquaint a researcher with a particular concept, to help group members remember events by group conversation, or to triangulate other findings (Denzin & Lincoln, 2000). In the case of this study, four focus groups were scheduled and conducted and participation in these groups was voluntary. The focus groups took between approximately one hour and two hours to complete. During this time, participants were asked questions on their participation in either the Nominating Committee training process, National FFA Officer selection process, or both depending on their level of involvement, using four different semi-structured research protocols. Focus groups occurred in four groups; the first was nine Nominating Committee members, the second was the three adult advisors to the Committee, the third was five members of National FFA staff, and fourth was a group of National Officer Candidates. The final two focus groups were used to assist in data clarification and triangulation. The focus groups were audio taped and transcribed in order to serve as the primary data source.
Secondary data consisted of field notes and supportive materials from the training and training transfer literature. Lincoln and Guba (1985) adopted the Glaser and Strauss (1967) constant comparative method for use in qualitative research. Typed transcripts were compared with the audiotapes for accuracy. Initial coding of the transcripts was done by the principal investigator. Following the development of
the categories, data reduction occurred resulting in themes and sub themes. The analysis was reviewed by another colleague in the same department.
Findings
Using the factors developed by Elangovan and Karakowsy (1999) and Noe (1986), the researchers have identified areas where improvement could occur increasing the likelihood of training transfer. As a reminder, the training environment elements being evaluated are: perceived relevance of training, choice in attending training, outcome expectancies, self-efficacy, and job involvement (Elangovan & Karakowsky, 1999); and knowledge acquisition and situation identification, job requirements and norms and group pressure, and rewards and organizational culture (Noe, 1986).
The more important and germane to a job situation training is believed to be, the greater the chance that the skills will be transferred. This is perceived relevance of training. Nominating Committee members provided mixed opinions on the various facets of the training. In terms of the materials received before the start of the National Convention, Nom Com members shared that the information was invaluable for building the context of the upcoming process. A participant noted “the prior preparation was essential for us as a collection of committee members to have a sense of ease when we came here as to what our back ground information was here to make the process move along during our time here.”
Still, in terms of the content of the prior materials, a common theme illustrated that while certain parts were useful, there was also a great deal of material that was irrelevant. A respondent stated “it was easier to just skim it and see that we knew our job and what our job was going to be, and just get the information that was going to pertain to us.” Another response was that “a lot of it we knew from being state officers. Just being in FFA we knew pretty much how to answer the questions without doing a whole lot of research.”
Many of the Nominating Committee members also shared some level of frustration with the delivery of the pre-Convention materials. This was evidenced by comments like “Prior to Convention I was disappointed. I need papers. I understand with everything online you can print it off and everything. But never did I receive something on paper saying anything…I didn’t realize we had a five week study guide until 5 days before convention, so I took those 5 days and worked on everything. But I wish maybe I got something in the mail so I could have had some hard documents.” And, from another participant, “I think the applications on line were ok, but it was hard to read them. They were really small
and condensed. I like to have material in my hand. I just like to have something to look at.”
The time line allotted for the pre-Convention preparation was also a matter of discussion among the Committee members, with general consensus being that more time is needed for the pre-Convention preparation. Members indicated that “if they can move that process [selection of Nominating Committee] up so that we can still start at the end of August. That would have made our lives a lot easier. It wouldn’t have been a 5 week cram.” An additional comment received was “that’s really a struggle I found is to find enough time to sit down and get everything printed off and read through it. Yeah, if it could have been mailed and go through and highlight through it, it would have been a lot easier.”
Once the committee members arrived at the National Convention and training continued face to face, committee members shared that the initial mapping out of the process and practical applications were beneficial; however, there were also portions of the training where relevance was lost. Feedback revealed “they did a great job of mapping out what our objectives were and how we were going to achieve them and at the end of the day they would show what we had been through and what we accomplished.” Again it was noted that “having the practice interviews were probably the most beneficial things that we did.” And, further, “I think that there might have been some time wasted on making sure we felt comfortable.”
From the perspective of the adults, the training needed to be more relevant and focused in one particular area – the videos demonstrating behavioral interviewing. It was stated that “there were lots of lessons to be learned. The training on Qualitative research needs to be done better to show the 1, the 3, and the 5.” In addition, a participant stated, “I think the videos would be more effective if first there had been a [live] demonstration of the 5, of a 3 and of a 1 and then some discussion on what they saw.”
When individuals have a choice whether or not to attend training, transfer is more likely to occur. In the case of the Nominating Committee, it was a unanimous sentiment that the training the received prior to the start of the National FFA convention was mandatory in order to secure your place on the Nom Com.
Outcome expectancies are defined as the belief that if individuals learn and use the training they receive it will bring the expected outcome. In the case of the Nominating Committee, several members discussed their feelings of the positive applicability of the training. One member indicated that “I don’t know why I was anxious, I just wanted to make sure I did a good job, but once that first one was
over I realized that the interview was exactly what they prepared us for during the practicing and things like that.”
Self efficacy is level of belief that an individual carries regarding their abilities to learn new skills and accomplish the goals of the training. The Nominating Committee members demonstrated a high degree of self efficacy. While some expressed a small amount of trepidation about the making the decisions asked of them, they all voiced the belief that they were indeed capable. This is evidenced in the statement: “Continuous encouragement that we were doing a great job and that we were doing what we came here to do. It felt really good knowing that we were succeeding, that we were growing to the outcome that we had to make this decision.” This was echoed by the Adult Consultants. “There were members of the committee who would have been stronger national officer candidates this year than those who were selected from their region.”
Job involvement or demand is the level at which an individual is expected to perform, or the amount of involvement required by the job. This is similar enough to the criteria of job requirements that these can be discussed together. While the Nom Com did acknowledge that the schedule of the selection is very rigorous, they also voiced concern that this level of rigor did sometimes impede their training transfer, particularly in reference to the Round Robin issues. However, they also were quick to point out that regardless of how rigorous the process was they felt sure that they would have slated the same people. “We went from being all day together watching the facilitation and then all of a sudden we were shifted into the round robin issues. Personally, I knew what was going on, but I wasn’t ready for the shifting in the environment. The start of the round robin issues was hard for me to get and try to understand what we were actually looking for.” Additionally, “We knew that they were going to be working long hours and they knew too that we were going to be doing that because we wanted to be fair and consistent for each candidate.”
In reference to the round robin sessions, the sentiment of the Nominating Committee members was echoed by the Adult Consultants: “Order of the rounds needs to change. For example, the Round Robin conversations need to occur earlier in the day while the Nominating Committee is still fresh. I became concerned about the scoring during the last few rotations because the committee members were obviously very tired and having a hard time staying engaged.” Further it was suggested “the expert rounds need to be broken up as well. They need to be more defined in what is going to go on, and how they will function.
There was so much dead time, and then things would go so fast that it threw the candidates and the Nominating Committee members off at times.”
When individuals can identify and demonstrate new skills learned they have positive knowledge acquisition. The members of the Nominating Committee shared that they could see the acquisition within themselves and each other; the further they traversed into the selection process. “I felt like I understood it [training content] further in the process…Things like in the training that I didn’t quite grasp it completely by the end of the road I had it completely grasped.” And comments such as “I think I knew it [training content] very well because they talked about it so I was able to hear it, they showed it visually through flip charts and videos and then I was able to actually apply it” were received from the participants.
As individuals come upon situations where skills and knowledge learned from training can be put to use, and those individuals can positively identify those situations, the likelihood of transfer is significantly increased. The Nominating Committee members demonstrated (via several avenues) their ability to identify not only the situations, but the nuances of their particular situations in relation to the scoring of each round of the selection process. “In the interviews we knew exactly what the three things we were to have…what a 1 was, what a 3 was, what a 5 was and how to recognize it.” “I think the easiest apart of the whole thing was just following exactly what it [training and score cards] said to do. I mean if you just looked at what you were supposed to look at, it was the easiest part.”
Norms and group pressure are defined as the common standards or practices developed by the group that shape a group’s social context. In the case of the Nominating Committee, the norms surrounded the solemnity of purpose of picking a National Officer team, something that each of them discussed with a great deal of respect. Also, each expressed a desire to be fair and consistent in the scoring to ensure that the selection process maintained a high level of integrity.
One individual said, “I know that as the first candidate was walking in I was trying to look at the score card…I mean as they were saying their first two answers, I was still looking at my score sheet, and I need to be looking at this and this and then I miss half of an answer or the whole answer. Trying to figure out what I was listening for and you know, mine probably got lower scores on that round because I could have missed something that they said that was actually really good, but I was trying to look at my score card.” Another indicated “I think it helped that everyone is professional on this committee. Including our facilitator, he really did a good job with how we were all going to be feeling at that point.”
Rewards, the positive outcomes gained when training is put into practice, influence the transfer of training. While there were no tangible rewards for their participation, unanimously, the members of the Nominating Committee voiced that their reward came with the opportunity to give back to an organization that gave so much to them. “We are people who have a lot of pride for this organization and I guess we are excited to have the opportunity to come back. It was a sense of being able to give back.”
Organizational culture includes the development of norms, in that culture is a major part of the development of the social context of the group. In the case of the Nominating Committee, a common theme with regard to organizational culture is the relationship that develops between National FFA staff, Nominating Committee Adult Consultants, and the Nominating Committee members.
Comments construct a snapshot of the role these individuals play within the process, and the effect that they have on the transfer of the training to the Nominating Committee members. Collectively, the Nominating Committee members imparted both positive and negative assessments regarding the treatment they received by the Adult Consultants. “A lot of times I felt like I was being treated like a little kid, but it wasn’t by National staff, it was by the other adult people.” It was noted that “the staff and facilitators were very different people in every respect, the way they handled us in the room, the way they talked outside the room. I realize that they are different people. I understand that, but they needed to be on the same page, which’s doing what, how the timing was going to go, when they are going to give us a break. I don’t think the adults were completely on the same page throughout the entire time.” “I think what was different for us was the person who was assisting us had been on National Nominating Committee back when he was in FFA so he knew what it was like.
He knew that we needed that critical time to review things, that it was critical that we did have time with the candidates. So he kept things light. His insights were some of the most valuable insights to anyone else that was here the entire week, for me personally.”
From the perspective of the Adult Consultants, their role was narrowly defined. “Our job is to work with the young men and women on the Nominating Committee. We provide them with strategic breaks when that is called for. We help them to re-center, and maintain the same level of intensity throughout the process.”
Conclusions and Recommendations
The training environment of the National FFA Nominating Committee does encourage the transfer of that training to job related situations. However, there are areas where changes can be instituted in order to advance training transfer and promote a positive learning situation.
Overall, the members of the Nominating Committee do see and acknowledge the importance of their training, which encourages training transfer. Yet, it should be noted that some of the training content and the delivery of the pre-convention training materials is a source of frustration which did impede, in some case, the transfer of that portion of the training. The researchers recommend that pre- convention materials be offered to Nominating Committee members in either hard copy or electronically. This will resolve the concerns of some that electronic copies were hard to read, too lengthy to print out, or were not received in time to be effective. Also, regardless of the delivery method, training materials should be clearly outlined and where content does not particularly pertain to the Nominating Committee, notations should be made. Finally, during conference calls, National FFA staff should follow up those notations with discussions that plainly outline the purposes for the provided materials.
When individuals have a choice whether or not to attend training, transfer is more likely to occur. In the case of the Nominating Committee, it was a unanimous sentiment that the training received prior to the start of the National FFA convention was mandatory in order to secure your place on the Nom Com. While the situation calls for mandatory meetings, the researchers would suggest more than one opportunity for the group to come together prior to National Convention. This will engage committee members when activities are less hectic, promoting a greater focus to the tasks at hand, instead of spreading the individuals thin during a time when a multitude of activities are taking place (beginning of fall semester). This variety could mitigate the mandatory nature of the Convention training and encourage transfer.
In the case of the Nominating Committee, several members discussed their feelings of the positive applicability of the training (outcome expectancies) which encourages training transfer. The National FFA staff should continue to provide Nom Com members with opportunities to practice the necessary skills prior to the selection process beginning. However, National Staff and Adult Consultants should evaluate the Nominating Committee members to demonstrate that the skills, when put into practice, do yield the proposed results. This will provide members with a sense of trust in the training content and further encourage transfer.
The Nominating Committee members demonstrated a high degree of self efficacy which encourages training transfer. While some members expressed a small amount of trepidation about the making the required decisions, they all voiced the belief that they were indeed capable. National Staff could use the aforementioned evaluation to further reinforce the confidence of the Nominating Committee members, increasing self efficacy and encouraging further transfer.
Job involvement or demand is the level at which an individual is expected to perform, or the amount of involvement required by the job. The Nominating Committee acknowledges that the schedule of the selection is very rigorous, and unlikely to change. They also voiced concern that this level of rigor sometimes impedes training transfer. In this case, the researchers recommend that the order of the rounds change to better accommodate the needs of the Nominating Committee. Rounds, post cut, should be ordered in such a way as to maximize the effectiveness of the Committee members’ evaluation strengths. Rounds where more focus is required (one-on-one interviews, expert rounds, etc.) should be scheduled early in the day. Rounds where all Nom Com members are present, or where content is not as technical, should be scheduled later.
Members of the Nominating Committee shared that they could see skill acquisition within themselves and each other the further they moved through the selection process. This skill acquisition will encourage transfer; nevertheless, steps can be taken to improve in this area. The researchers recommend that the National FFA staff and Adult Consultants be vigilant in confirming the knowledge and skill acquisition. A pre- and post-test format would be useful in confirming the skill acquisition, observational data can be used to confirm the demonstration of new skills, and an analysis of the variance of scores would confirm the consistency of the human instruments. Another recommendation would be to continue the positive reinforcement of the Nominating Committee by the National Staff and Adult Consultants.
Nominating Committee members demonstrated that they could identify situations where skills learned in training could be put to use, which encourages training transfer. Yet, there are ways that this area could be augmented by taking a few very simple actions. National Staff should continue to provide concrete scenarios that mimic the selection process prior to the beginning of the first round. These scenarios can be evaluated in order to provide positive and negative feedback.
Specifically, these scenarios should be related to the behavioral interview rounds where the nuances of scoring are more easily seen live, than explained via other avenues.
When groups develop norms, and positive group pressure is encouraged, training transfer is increased. In the case of the Nominating Committee, they developed norms early, and held to their norms throughout the process. National staff should continue allowing the Nominating Committee to develop their own norms and ways in which to hold each other accountable.
While there were no tangible rewards for their participation, unanimously, the members of the Nominating Committee voiced that there reward came with the opportunity to give back to an organization that gave so much to them. National Staff should discuss these intangible rewards with the Nominating Committee members in order to bring further clarity to the essential nature of the service that they are providing to the FFA Organization.
Organizational culture includes the development of the social context of the group. In the case of the Nominating Committee, a common theme with regard to organizational culture is the relationship that develops between National FFA staff, Nominating Committee Adult Consultants and the Nominating Committee members. These relationships have an effect on the transfer of training. In relation to the National Staff and volunteers, the Nominating Committee identified a positive relationship which encourages training transfer. These positive relationships should be encouraged and nurtured through the development of group norms at the outset of the training process. However, the relationship defined by the Nom Com with their Adult Consultants may hinder training transfer. In this case the researchers recommend that the National Staff work with the Adult Consultants independently of the Nominating Committee in order to concretely define the role within the process, as well as discuss personal and professional boundaries within which they will work with the Nominating Committee. Together, National Staff should openly address the individual roles of each group, confirm appropriate boundaries of all parties, and for all individuals involved provide an avenue to report feelings of discomfort or situations where boundaries have been crossed so that they may be addressed immediately by National Staff.
References
Dentin, N., & Lincoln, Y. (Eds.) (2001). Handbook of qualitative research (2nd ed.). London, Sage Publications.
Dormody, T., & Seevers, B. (1994). Participation of FFA members in leadership development activities: A tri-state study. Journal of Agricultural Education, 35(4), 42-48.
Elangovan, A. R., & Karakowsky, L. (1999). The role of trainee and environmental factors in transfer of training: An exploratory framework. Leadership and Organization Development Journal, 20(5), 268-276.
Erlandson, D. A., Harris, E. L., Skipper, B. L., & Allen, S. D. (1993). Doing naturalistic inquiry. Newbury Park, CA: Sage Publications.
Garavaglia, P. L. (1993). How to ensure the transfer of training. Training and Development, 43(10), 63-69.
Glaser, B. G., & Strauss, A. L. (1967). The discovery of grounded theory.
Hawthorne, NY: Aldine.
Lincoln, Y., & Guba, E. (1985). Naturalistic inquiry. Thousand Oaks, CA: Sage. National FFA. (2006). Retrieved on October 15, 2006 from: http://www.ffa.org/.
Noe, R. (1986). Trainees attributes and attitudes: Neglected influences on training effectiveness. Academy of Management Review, 11, 736-749.
Patton, M. Q. (1990). Qualitative evaluation and research methods. Newbury Park, CA: Sage.
Biography
Dr. Jacklyn Bruce is an Assistant Professor of Youth and Adult Leadership Development at The Pennsylvania State University in the Department of Agricultural and Extension Education. In this position, Dr. Bruce provides leadership for a variety of state level leadership development programming for the Pennsylvania 4-H program, maintains a rigorous research program in the area of transfer of leadership training and skills and an outreach program on working with teens for Pennsylvania’s 4-H Youth Development Extension Agents.
Dr. Kristina Ricketts is an Assistant Professor of Leadership Development at The Pennsylvania State University in the Department of Agricultural and Extension Education. In this position, Dr. Ricketts provides direction for the resident education program in leadership development and maintains a rigorous research program in the area of community and rural leadership.