An impact assessment provides information about the impacts produced by an intervention, whether positive, negative, direct, or indirect. Properly done, it can improve and develop the effectiveness of a project.
The growing interest in impact assessment has raised several questions about the process.
Before getting into the subject of assessment, here is an introduction to what social and environmental impact initiatives are:
“Social and environmental impact initiatives have as their purpose the resolution of a problem that afflicts society or a community. In addition to the challenge of becoming part of the solution to a given problem, these initiatives are constantly challenged to prove the impact they generate in the contexts in which they operate.”
Paulo Cruz Filho, 2018. Social Entrepreneurship and Social Innovation in the Brazilian Context
What is impact assessment?
Impact assessment measures the changes that have occurred in a community, population, or territory from certain interventions.
These interventions can be small or large projects, sets of activities, or public policies.
According to the Organisation for Economic Co-operation and Development – Development Assistance Committee a technical definition of impact is:
“positive and negative, primary and secondary long-term effects produced by a development intervention, directly or indirectly, intended or unintended.”
(OCDE-DAC 2010)
This means that an impact assessment goes beyond measuring and describing the impacts caused, it tries to understand what role the intervention played in these outcomes, whether intended or unintended.
Why assess Impact?
An impact assessment can have several purposes, it all depends on its purpose and who the stakeholders involved are. Some common reasons for assessment are:
Accountability
- Demonstration, disclosure, and transparency of the work done.
- Accountability: how are the resources obtained applied, and what are the “results” obtained?
- Listening and informing the community about how a project is benefiting them.
- Communication and dialogue with donors, investors, partners, government (edicts), society, clients, citizens.
Management Tool
- Professionals in the social sector:
- Increasingly specialized and rigorous employees.
- Increasing presence of auditors and evaluators.
- Tool for management and strategic decision-making.
- Development of new opportunities:
- Obtaining potential investors.
- Establishing partnerships.
- Learning how to replicate a pilot project.
- Learning how to adapt a successful intervention to another context.
Performance Evaluation
- Evaluation of program performance and the internal and external performance of the organization.
- Improving the products and services provided.
- Deciding whether a project should be continued or expanded.
When to assess the impact?
An impact assessment should only be done when its usefulness is clearly defined and the timing of the intervention to be studied must be taken into consideration. It is important, however, to structure the assessment before the initiative begins so that there is enough time to collect important data and to be able to measure the “before and after”.
Consideration should be given to the relevance of the assessment to organizational strategy, its potential utility, and the commitment of stakeholders to use its findings.
Timing should also be taken into account. An early assessment will provide an inaccurate picture of impacts, which may not have had sufficient time to develop. A late assessment may not arrive in time to influence decisions following the project.
Who participates in the impact assessment process?
An essential step in effective impact assessment is the choice of who will participate in the process. Being clear about the purpose of each participation is essential for managing expectations and guiding implementation.
With an initial analysis of participation you can, for example, ensure that the people impacted by the project have a voice in representing the results rather than being represented by an external evaluator’s indicators.
The participation decision can be both pragmatic and ethical. Pragmatic in the sense of seeking better data and information. Ethical in the sense of giving a voice to those who will be affected by the decisions.
The beginning of a participatory assessment should answer the following questions:
- What is the purpose of stakeholder participation in this assessment?
- Who is participating?
- When and why?
Theory of Change
The Theory of Change is an impact management tool, but it can also be used in assessment. It allows you to deeply understand what your immediate impact is, what the medium-term outcome will be, and the long-term impact. With it you can carry out organizational planning and understand the contribution being made.
The Theory of Change has 5 elements.
- Inputs: Resources and inputs needed to operate the project, program or initiative. These can be financial, material, human, technological, etc.
- Activities: All actions performed. Activities (interventions) carried out, products and services offered. Modus operandi and articulation with the target audience.
- Outputs: Immediate effects resulting from the activities performed. Tangible and immediate reach of the products/services offered. Immediate benefit to the target audience (beneficiaries).
- Outcomes: Changes expected to be generated in the target audience and in the lives of the beneficiaries.
- Impact: Desired structural impact on the ecosystem. Changes generated in society, in the social system.
For example, what would a Theory of Change for a school for children with disabilities look like?
- Inputs: Teachers, classrooms, chairs, desks, environments and teaching materials.
- Activities: Teacher-student relationship, the class.
- Outputs: Did the child learn what was taught?
- Outcomes: Does what the child has learned continue to be performed? Has it transformed his or her life?
- Impact: Has the child been included, that is, is he participating more in society?
The Theory of Change allows you to carefully craft the alignment between the purpose of the initiative and the desired and achievable social and environmental impact.
Impact Assessment Methods
When conducting an impact assessment, we can have several levels of complexity.
At a first level, we have a simple assessment of indicators before and after the intervention. For example: when carrying out a project with homeless people, we analyze indicators of well-being and health, family ties, and employment. How the beneficiary was before the project and how he or she is after.
A second level of complexity occurs when the methodology already has a grid of indicators designed. For example: System B, SDGs (Sustainable Development Goals), and GRI (Global Reporting Initiative). At this level we have the advantage of comparing indicators between other organizations.
At a third level of complexity we introduce the concept of a control community. That is, we select two similar and nearby communities, in one of them we carry out the project and in the other we do not. At the end, we compare indicators and perform the assessment in both communities to understand the impact of the action.
Finally, the fourth level contemplates the monetization of the measured results, that is, it consists in transforming the measured impact into monetary value, be it an output, an outcome, or a long-term impact (e.g. SROI)
Which impact methodology do I choose?
This answer depends on some factors, such as
- Purpose of the impact assessment and measurement process.
- Size of the organization and available team.
- Complexity of the project or initiative.
- Level of depth expected.
If you have the resources and staff to conduct the analysis, you can start at the most advanced level. If it is your first time measuring impact, start at the simplest level and move forward as the project develops. The important thing is to get started!
Do you have any questions or need a professional assessment?
