Recently there was a discussion on AEA's listserv EvalTalk on what it means to think evaluatively. Multiple persons weighed in on this topic, but it surprised me how few persons seemed to understand the idea of using evidence versus using evidence to make an estimation of merit, worth , or value. In essence, it's the same conundrum I've found in purported evaluation reports that detail "what is" versus going one step further and addressing "what is the value (of what is)".
However, a few people did, as it seems to me, accurately identify what is meant by thinking evaluatively. One example I really liked was written by Eileen Stryker, who wrote, "It seems to me that thinking evaluatively is about how we arrive at or account for judgments about value or quality. Evaluative thinking means being tuned in to value judgments people make (e.g., listening for such language as: that's good, he's doing a good job, the program is working, they're really getting better, and words like effective, quality, good, bad, better, improving, etc.), and questioning how those judgments were arrived at and what evidence may exist to substantiate the value claim."
Eric Weir strongly agreed with Eileen's definition and added: "[Evaluative thinking] is either
using evidence to support value judgments or assessing the extent to which value judgments are supported by evidence.....Evidence does not support or undermine judgments by itself. Arguments are needed to connect the evidence to the judgment."
Last, Bob Williams summed it up (nicely) as "evaluative thinking is the informed judgment of value, merit or worth".
This was an incredibly interesting discussion and made me wonder a few things:
Why do we, as evaluators, seem to so often feel insecure in evaluatively assessing outcomes?
What does this say about evaluation training and the need for more emphasis on training to think evaluatively?, and
Are there differences in how evaluatively persons think based on training, evaluation field, years of experience, etc?
Would love to know what others think about these questions. Feel free to comment!