Terms of Reference for an End of Project Evaluation for G.R.A.C.E Africa's New Partners Initiative ProgramProgram title:- Engaging
a Network of Indigenous Youth Groups and Community Based Organizations
in the Support of Orphans and Vulnerable Children”.Grant No:- GHO-A-00-09-00004-00
Evaluation period: December 1st, 2008 - September 30th, 2011
1.0 BackgroundGrassroots
Alliance for Community Education (G.R.A.C.E) Africa is a non-profit
organization registered in 2001 under Kenya’s NGO Act.
The
organization’s mission is to enhance the capacity of community-based
organizations for self-determined, high impact and sustainable
initiatives leading to better health and development.
G.R.A.C.E’s
key approach to development work entails strengthening the capacity of
local grassroots organizations to design and implement programs aimed at
serving poor and marginalized groups and communities.
Since
December 2008, and with support from the United States Agency for
International Development (USAID) through the New Partners Initiative
(NPI), G.R.A.C.E has been implementing a three year program in three of
Kenya's eight provinces namely Nyanza, Eastern and Central.
1.1 Purpose of the NPI ProgramThrough
the NPI Program, G.R.A.C.E contributes to the strengthening of
community based responses to the plight of OVC as well as the prevention
of HIV in Kenya.
This is in line with the Kenya National
HIV/AIDS Strategic Plan priority area on mitigation and social impact,
whose key focus priority areas include: strengthening social mechanisms
for orphan care; ensuring access to social services by OVC (including
food & nutrition, education, health, shelter & care, protection
and supporting OVC caregivers to provide quality care to OVC under their
care); and strengthening the legal and policy framework for protecting
the rights of OVC in the country.
1.2 Program Geographical Areas of CoverageNyanza Province: Mbita, Siaya, Migori, Kisumu town, Borabu and Oyugis
Central Province: Kerugoya, Kiambu, Kikuyu and Kamahuha
Eastern Province: Isiolo, Maua, Chuka and Embu
1.3 Program Goal:The goal of NPI program is to improve quality of orphan care in the targeted areas.
1.4 Program Objectives:The following are the strategic objectives of the NPI program:-
- Provide
Early Childhood Education and related services to 1,800 orphans and
vulnerable children aged 3 to 6 years by enrolling them in ECD centers.
- Conduct
skills training workshops to 1,200 OVC caregivers and community members
on psychosocial support, entrepreneurship, HIV Prevention and food
security.
- Build and strengthen the capacity of 15 sub partners
for quality programming through institutional strengthening
assessments/initiatives and provision of technical assistance.
- Increase
information sharing and referral systems by participating in key
national fora and hosting regional meeting with stakeholders.
- Provide HIV prevention messages to 20,000 community members.
1.5 Program activities and implementing partners
NPI
program activities include provision of education and related services
to OVC aged 3-6 years in community ECD centres; strengthening the
capacity of caregivers and other vulnerable community members in
economic well being, Child Protection, Psychosocial Support, food and
nutrition; and promoting HIV Prevention through public education and
Behaviour Change Communication (BCC).
The program is implemented
in collaboration with four organizations namely the Trust for Indigenous
Culture And Health (TICAH), UZIMA Foundation, Population Services
International (PSI) and SAHAYA Deaf Kenya.
These partner
organizations have been engaged by G.R.A.C.E to provide skills and
technical support supervision to the 15 local Community Based
Organizations that implement the program at the various target
districts.
Several line ministries and government departments
have also supported the implementation of the program mainly at the
district level. These include the ministries of Education, Agriculture,
Public Health and Sanitation, Gender and Children’s Services and the
local government.
2.0 Purpose and Scope of Work for the End Term Program Evaluation
2.1 Purpose and rationale for the assignment
The
NPI Program is scheduled to come to an end at the end of November 2011.
In this regard, G.R.A.C.E is sourcing for an external consultant or
consultancy firm to undertake the end of program performance evaluation
in the course of October, 2011.
The overall purpose of the end of
program evaluation is to measure the project outcome of the strategic
objectives; determine the relevance, efficiency and effectiveness of the
interventions; and highlight program achievements and gaps.
In
addition, it will gauge the level of community and other stakeholder
participation in and ownership of program implementation processes; the
degree of sustainability for the various program interventions;
highlight and document promising practices, key challenges encountered
and lessons learnt for policy influencing or future programming by
G.R.A.C.E and other agencies undertaking similar work.
2.2. Evaluation Objectives
The specific objectives of this evaluation will be to assess the following:
1. Outputs
and outcomes: Outputs and outcomes generated by the program in relation
to the stated goal, objectives and desired results. In particular, the
evaluation will assess the effects of the program activities on the
targeted beneficiaries including the OVC, their caregivers, partner
CBOs, and community members reached with the various services.
2. Relevance
- whether the program interventions met needs of the beneficiaries; the
appropriateness of results in relation to the needs of the communities,
national policies and priorities.
3. Effectiveness:
In particular assess the extent to which program interventions achieved
the desired outcomes, factoring in issues of program management
including decision making processes, risk management, institutional
arrangements and partnerships and their effect on the program results.
4.
Efficiency - the relationship between the quantity, quality, and
timeliness of program inputs, including personnel, consultants, travel,
training, office equipment and financial sub grants to partner CBOs.
In addition, determine the quantity, quality, and timeliness of the
outputs generated and whether the resources were spent as economically
as possible.
5. Sustainability
– assess the readiness of partner CBOs and other stakeholders to
sustain program interventions, in particular assess the infrastructure
and systems of partner CBOs, resources available to sustain the
activities and services, collaborative links and referral networks with
other service providers, and the level of community ownership.
6.
Program Improvement areas: capture key successes, best practices
lessons learnt, implementation challenges, constraints, strengths and
weaknesses and provide recommendations for possible scale up or
replication of the program in totality or in part.
2.3 Evaluation Approach
The
evaluation will follow the G.R.A.C.E and USAID Norms and Standards.
This implies inter alia that the evaluation must be guided by the
principles and goals of community empowerment and focus on Orphans and
Vulnerable Children.
Although the consultant will be expected to
come up with a comprehensive evaluation design for the exercise, the
following principles will be relevant while considering the methodology
of the evaluation:
a. Use of participatory approaches including children to measure performance.
b.
Evaluation processes should engender greater accountability,
transparency, and help build capacity, attempt systematic and objective
assessment(s), guide decision-makers and/or program managers and provide
information on whether underlying strategies and assumptions used in
program implementation were valid (what worked and what did not work and
why)
c. The measurement of effects/benefits of program interventions;
d. Giving stakeholders the opportunity to clarify issues related to program delivery
2.4 Evaluation questions:
The following broad questions will need to be addressed:
- Were
all program activities implemented as planned? In cases of deviation
from the original plans, what were the reasons for deviation?
- Were
program activities and interventions appropriate and in line with the
National and USAID policies or guidelines in the various service areas?
- Were the activities relevant, and were they implemented in an appropriate, effective and efficient manner?
- What are the key program outputs and outcomes, and to what extent did the program activities contribute the outcome?
- To what extent did external factors influence the outputs and outcome of the program?
- What capacity building activities were undertaken to strengthen implementing partners and how did they contributed to:
- Project success or hindered progress,
- Organizational growth in general,
- Were there any lessons learned about what should have been done differently in the capacity building interventions.
- To
what extent did G.R.A.C.E coordinate with both in-country USAID team
and Government of Kenya (e.g. partner meetings, membership on technical
working groups, involvement on government working groups, participation
in trainings, provincial government meetings and coordination, etc.).
- What referral networks did the project develop or work with?
- Are the results sustainable? (Will the outputs and outcome(s) lead to benefits beyond the life of the existing NPI program?)
- What are the notable achievements, challenges, promising practices and lessons learnt from the program?
- How
can G.R.A.C.E. do things better in the future? (Which findings may have
relevance for future programming or for other similar initiatives
elsewhere?)
2.5 Evaluation Methods
The
evaluator is expected to employ a variety of data collection and
analysis techniques for both quantitative and qualitative data to ensure
a comprehensive evaluation exercise.
In essence, the process has
to factor in the participation of children who either benefited from or
took part in program activities.
The key methods for the exercise will include but will not necessarily be limited to the following:
- Desk
review of relevant program documents (these include the program
proposal, annual work plans and budgets, program progress reports,
minutes of technical meetings, progress reports from CBO partners and
other documents related to the NPI program);
- Review of the data
collected by G.R.A.C.E staff and CBO partners using the Child Status
Index (CSI) and other routine monitoring tools.
- Review of the
G.R.A.C.E database as well as other databases and registers kept and
maintained by the CBO partners at their offices and the various ECD
centres supported by G.R.A.C.E.
- Interviews with various
informants including G.R.A.C.E management team, Program and Finance
staff, staff of partner CBOs, opinion leaders, government officials
involved in project implementation, caregivers and child beneficiaries.
- Observation and focus group discussions.
3. Key Responsibilities of the Consultant
3.1. Tasks and Responsibilities
- Work
with the G.R.A.C.E Program team to develop an appropriate evaluation
design and data collection tools based on indicators as documented in
the program proposal and performance monitoring and evaluation
framework.
- Develop a Detailed Implementation Plan (DIP) for the evaluation process.
- Review
key program documents including those produced by program partners;
such documents shall include but are not limited to the background
project document, annual work plans, progress reports, and other
documents related to the Project.
- Work with the G.R.A.C.E
Program team and implementing partners in carrying out the necessary
field activities for the end term evaluation, including a pre-testing
and refinement of the developed tools and data collection.
- Conduct
data analysis as appropriate and generate a draft evaluation report
detailing evaluation methodology, process, key achievements, challenges,
lessons learnt, promising practices etc.
- Present the draft evaluation report to G.R.A.C.E for input, discussion and revision if need be.
- Review
the draft report to include feedback from G.R.A.C.E and her partners,
and prepare and submit three hard copies and one soft copy of the report
to G.R.A.C.E.
Deliverables
- An end of term program evaluation design detailing the methodology, tools and approaches to be used in the exercise.
- A Detailed Implementation Plan for the evaluation exercise.
- Three (3) bound hard copies of the end term Evaluation Report.
- One soft copy of the End-Term Evaluation Report in a CD.
- The consultant will also be required to submit to G.R.A.C.E. NPI Project all study materials including:
- Soft copies of all data sets both quantitative and qualitative
- All filled quantitative data collection tools and qualitative data recording materials.
- Any other non-consumable documents/items that will be used in the course of the planned consultancy
The
contents of the final report will be in line with USAID Evaluation
Policy and as agreed upon together with the G.R.A.C.E Management Team.
However,
the evaluation report must include any significant unresolved
differences of opinion on the part of funders, implementers and/or
members of the evaluation team where those arise.
4.0 Timeframe
The assignment is expected to take place between October, 3rd and October 29th 2011 with a maximum of 20 consultancy days
Consultant’s Profile: Skills, Experience and Qualifications
- Proven
experience in research, monitoring and evaluation of non-governmental
community development structures, process and integrated programs.
- Sound knowledge and understanding of programs related to OVC support and HIV Prevention.
- Sound
knowledge and understanding of OVC service provision national
standards, health sector policies and systems, and HIV&AIDS response
framework in Kenya.
- The Lead Consultant must possess a Masters
degree from an internationally recognized university and at least 5
years experience in programming at a senior level, as well as experience
in evaluating complex programs related to OVC, HIV and AIDS.
Other skills necessary include:
- Quantitative and qualitative research methods
- Program analysis, evaluation and multi-districts/community study design
- Knowledge on gender main streaming, gender analysis and cross-cultural studies
- Organizational development and systems strengthening
- Data analysis and report writing
- Strong communications skills -oral, written and presentation skills
5. Expression of InterestInterested
consultants or firms are requested to submit an Expression Of Interest
detailing their interpretation of the TOR, proposed methodology, work
schedule and proposed budget (in Kenya Shillings), a capability
statement and copies of the relevant Curriculum Vitae.These should be submitted on or before the 29th of October 2011 to the following email addresses;Email: pascal.mailu@gmail.com and, onyaloj@gmail.comHard copies should be addressed to The Program Coordinator,G.R.A.C.E. Africa,P.O.Box 13993 -00100,Ole Odume Road, 2 doors past Riara Road,Nairobi, Kenya.Tel +254 (0) 20 387 2856.
