As part of the Evaluation and Assessment competency, I’ve started reading How to Be Involved in Program Evaluation: What Every Administrator Needs to Know.
As I’m just starting to get a feel for how best to meet this competency, I’m reading this book fairly slowly.
In the first chapter, McNeil et al. compare the differences between the program director and the evaluator (p. 17). Some of the ones that caught my attention are:
Program Director / Evaluator
Person of immediate action / Person of deliberation
Pragmatic / Realistic
Concerned with current program / interested in replicable results (p. 17).
So far in my 10 years or so of working at the ISD, I’ve only really tried to run my program and evaluate it myself. Only occasionally, usually at the end of a grant, have my supervisors brought in an outside evaluator. Some of the evaluations I do are:
- Collecting feedback on MysteryQuest sessions
- Collecting data and creating end of the year reports
- Collecting evaluation data at the end of my online classes
What I’m realizing is, when I run my own evaluations, I may be inadvertently missing important information, not hearing from all the participants, and possibly looking only for data that supports my program. There may be a bias.
In some cases, the program director is highly involved in evaluation and guiding it, which these authors see as a good thing (p. 32). However, involved directors need to remember that “others involved with the program must become active stakeholders in the evaluation process.” Hmm. I’m not sure yet what that means for my own practice, but I realize I need to hear from others. Who are the others? How do I hear from them? How do I involve them without burdening them?
I’m definitely seeing that using a known model to guide a person’s work makes it much less likely that an important piece will be overlooked.