When we want to understand whether a program is actually helping people, we often hear these terms: performance measurement, program evaluation, impact evaluation, internal audit, and research. They can feel confusing because they sometimes use the same tools (surveys, interviews, data, and charts). But they exist for different reasons and answer different questions.
Before defining program evaluation, it helps to clarify what it is not:
Sometimes organizations collect data out of curiosity or interest and report selectively on what looks good or convenient. While that may be called “measurement,” it is not yet evaluation. Without a systematic approach and clear questions, the findings lack credibility and decision-making value.
Performance measurement focuses on monitoring progress and operational efficiency. It tells us what is happening right now. Key Performance Indicators (KPIs), dashboards, Balanced Scorecards, and Continuous Quality Improvement approaches fall under this category. They help managers adjust programs in real time and track whether activities are being delivered as planned.It answers:
- Are we delivering what we said we would?
- Are we meeting our targets?
- Where are operational bottlenecks?
Now, we can define what program evaluation is:
Program evaluation is a much broader concept. It asks whether an initiative is well-designed, well-implemented, and achieving its intended results for learning and decision-making. It’s a systematic inquiry (both qualitative and quantitative) into the question: Is this program effective, and how can it be improved? As you can see, program evaluation occurs at every stage of the program: planning, design, implementation, and outcome achievement. Evaluation during implementation may look similar to performance measurement, but it goes further by asking whether the overall logic and strategy make sense.
However, chances are, as a social entrepreneur, you might have heard a particular type of evaluation more than others: impact evaluation. It usually occurs at the outcome achievement stage and asks about causality: Did the program cause the outcomes, and by how much? We often use ‘Cost-benefit analysis’ or ‘Experimental Evaluation’ to do Impact Evaluation. A version of this is called ‘Impact Measurement’, which sometimes focuses less on causal attribution and more on overall contribution. ‘Social Return On Investment (SROI) is a popular impact measurement approach.
On the other hand, internal audit does a very different function. It is an assurance function focused on risk, controls, compliance, and governance: “Are we doing things right and safely, according to rules and good control practice?” While impact evaluation focuses on results, internal Audit is increasingly important for risk management.
Lastly, social research is broader knowledge generation about social phenomena and behaviour; it may inform programs, but it’s not necessarily tied to a specific initiative or decision cycle. We also use Market or User Research to understand similar programs and target audiences.
To bring these terms together, let’s consider the the Airport Example.
Imagine driving to the airport.
- Performance measurement is your dashboard and GPS, speed, fuel level, ETA. It tells you what is happening right now so you can adjust.
- Program evaluation is stepping back afterward and asking whether you chose the right route, departure time, and preparation strategy, and what you would change next time.
- Impact evaluation zooms in on a specific decision: Did taking the toll road actually get you there faster than the free route?
- Internal audit checks safety and compliance, license, insurance, and maintenance.
- Research is broader knowledge about traffic patterns, congestion, and commuter behaviour.
Each serves a different purpose:
- Performance measurement helps you manage day-to-day operations
- Program evaluation helps you learn and make strategic decisions
- Impact evaluation helps you answer whether your intervention caused a change
- Internal audit protects the organization
- Research ensures your design is grounded in reality
They may use similar tools, but they answer different questions!
As you can see, there is a use for all of the above in different contexts. Performance measurement helps you manage a program day to day and spot problems early, while program evaluation helps you step back, learn what should be improved, and decide whether the program should be expanded, redesigned, or even stopped. Impact evaluation goes one step further by helping you answer the tough question funders and partners often ask: “Did it really work, and did our program actually cause the change?” Internal audit protects your organization by checking that money and processes are handled safely and properly, reducing financial, legal, and reputational risks. And research, including user and market research, helps you design programs that match real needs and experiences instead of relying on assumptions.
Check back in next week for the second question in this AMA series! If you have your own question to ask Ashraful, send it along to hello@hubottawa.org. Or if you’re interested in learning more but have specific questions about your organization or endeavour, book a 1:1 coaching session with Ashraful!
The Coaching for Impact program is sponsored by TD Foundation!