Reflection on Evaluator Competencies

Based on what I have learned this week through the readings and the live Zoom session, and on a scale of 1 to 6, I would rate myself a 1 as an evaluator. The reasoning behind this rating is my extremely limited experience with evaluations, but especially my lack of knowledge pertaining to evaluations in relation to LIDT. Nearly all of Stevahn's competencies were new to me, though many of them made logical sense.

In accordance with Stevahn’s competencies, I am strong in “Professional Practice, Remain open to the input from others, Interpersonal Competence, Use written communication skills, Use verbal/listening communication skills, Facilitate constructive interpersonal interaction, Demonstrate cross-cultural competence.” I have gained these skills through my current position as a library aide, but also through my Bachelor’s degree in English literature and my extensive background in reading and writing. Working in a school has given me a wealth of opportunities to practice communication skills, and learn empathy and understanding toward different cultures and how those cultures influence an individual’s learning.

All of the other skills are competencies I have not had the opportunity to put into practice yet, especially, “Applies professional evaluation standards, Contribute to the knowledge base of evaluation, Knowledgeable about quantitative/qualitative methods, Specifies program theory, Assesses validity/reliability of data, Budgets an evaluation, Identifies needed resources for evaluation, Trains others involved in conducting the evaluation,” to name a few (Stevahn et al. 2005). Some of these skills give me a bit of anxiety, as I do not have any experience creating a budget for evaluations, or even a small knowledge base from which to draw from previous experiences. However, I have been studying learning theories in my last three classes, and I do understand a little about quantitative and qualitative approaches. Therefore, I do have some background to pull from, and there have been other portions of LIDT that have caused me some anxiety, but by breaking the whole into smaller, bite-sized pieces, I can now affirm I have so many more LIDT skills than I did when I started this program. I have much more confidence in my abilities, and am excited to continue learning. Consequently, I know I am capable of learning and applying difficult concepts, and I can master them with practice.

I also enjoyed learning about the CIPP model of evaluation, especially questions we can ask ourselves in the formative and summative stages. I have faith that this is something I can do, and it goes hand-in-hand with design, because as designers we must always be examining the formative questions of our designs when we have something we need to teach: What should we do to teach it? How can we teach it? Is our design actually teaching it? Did it work? Then, we should also be asking ourselves the summative questions: How did we address the need? Was the design created and executed well? Was it successful? Asking these questions brings us from the level of dime-a-dozen designers to cutting-edge experts (Calhoun, Sahay, Wilson 2020). Though we may not become professional evaluators after this class, it is necessary to have evaluation skills so we can apply them throughout our design process.

Some of the things that surprised me were that an evaluator’s role is quite challenging, as often the members of the organization are skeptical of his or her expertise, and resistant to change. The evaluator, therefore, must not only have a high level of understanding of the program and how it should operate, but also have incredible communication and diplomacy skills to bring the employees together in unity in regards to the program. The evaluator needs the skillset to demonstrate to the stakeholders the need for the evaluation, and also to present the data in a way that shows what changes need to be made. I did not realize the lengthy list of skills an evaluator must possess (Russ-Eft & Preskill 2024).

Another surprising skill an evaluator must possess is the ability to “observe legal standards” (IBSTPI 2006). Being aware of legal standards would take time and effort, and also puts an evaluator in a bit of a compromising position if he or she is not up-to-date with all legal ramifications. Much is required of an evaluator, and it seems to be a stressful role. This part of evaluations sounds complicated, but not impossible.

The evaluator must have a high level of cross-cultural sensitivity. This fact did not surprise me as much. This is also a skill required to be a learning designer. However, it was interesting to note that evaluators must be aware of not only cultural influences in companies and programs, but also political influences as well. The laundry list of competencies seems a mile long. Again, it is not a role to be taken lightly.

If I am being completely honest, evaluation is not a skill I am interested in pursuing. I did not connect with the readings, and it will be challenging for me to remember and comprehend the competencies required for skilled evaluators. However, I can see that learning designers cannot overlook this crucial skill. Within the competencies necessary to become a professional evaluator are also competencies necessary to be an expert designer: interpersonal skills, ability to analyze data and apply it to improvements in the design, empathy towards different cultural and learning backgrounds, program management, and the list continues. Therefore, even though this class is going to be a challenge for me, and it is going to be a real stretch for me to learn these skills, I do see their need and why and how they will make me a more professional and marketable designer. 

References

Calhoun, C., Sahay, S., & Wilson, M. (2020). Instructional Design Evaluation

International Board of Standards for Training, Performance, and Instruction. (2006). Evaluator Competencies. www.ibstpi.org

Stevahn, L., King, J. A., Ghere, G., & Minnema, J. (2005). Establishing essential competencies for program evaluators. American Journal of Evaluation, 26(1), 43–59.