Consultant for Project Evaluation by Using Outcome Harvesting Methodology

Terms of Reference

Project Evaluation by Using Outcome Harvesting Methodology

Project Title: Initiative for Indigenous Resource Enhancement and Development (I-READ)

Overview:

Forum for Language Initiatives is planning to hire a consultant to work with the FLI MEL lead to conduct an external end-of-project evaluation of the “Initiative for Indigenous Resource Enhancement and Development (I-READ)” funded by the PMU Sweden and its impact on the training of Indigenous languages speakers and promoting and documenting indigenous languages of Northern Pakistan (Gilgit Baltistan, Azad Jammu & Kashmir and Khyber Pakhtunkhwa). The evaluation includes reviewing internal and external documents, conducting key informant interviews (KIIs), and focus group discussions (FGDs) with the project team, beneficiaries, and key stakeholders.

1.  Introduction of the Organization:

The Forum for Language Initiatives (FLI), established in 2002, is a registered non-profit organization dedicated to the preservation and promotion of endangered languages of Northern Pakistan. FLI\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\'s mission includes building the capacity of native language speakers to document their languages through training, workshops, and mentorship programs. FLI also publishes Indigenous materials in and about these languages and advocates to use of the languages in public and private institutions, especially educational institutions. It facilitates knowledge sharing among diverse ethnolinguistic communities in the northern region of Pakistan. Moreover, FLI supports communities in resolving their orthographic and digital issues and assists provincial governments in implementing language-related education policies.

2.  Project Outcomes:

The project aspired to achieve the following five Outcomes:

1)      Governments (Federal & Provincial) bodies recognized the ethnolinguistic rights of the indigenous people of northern Pakistan.

2)      Increased women's awareness on their rights and entitlements.

3)      Government/NGOs provided a conducive environment for learning in mother tongues in primary schools of Khyber Pakhtunkhwa & Gilgit Baltistan.

4)      Empowered Language Communities of Northern Pakistan to protect and promote their linguistic and cultural identity.

5)      Enhanced capacity of FLI (organisation & staff) to achieve the organisation's mission and vision and programmes/projects expected outcomes.

3.  Purpose and Objective of the Evaluation:

The evaluation aims:

a)       To assess project activities and results and develop recommendations for FLI on improving the impact of our work.

b)      To assess the intended and unintended outcomes of the project and to determine the level of the project’s contribution to these outcomes.

c)       To identify and analyze lessons from FLI approaches of work with Indigenous Language Communities of Northern Pakistan.

The specific objectives of the evaluation are to:

a)       Identify and describe the outcomes (both expected and unexpected) from the project implementation and their significance in the project context.

b)      Establish the project\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\'s contribution (direct or indirect causal link) to the outcomes.

c)       Identify lessons learned and best practices related to the project context, actors engaged, strategies applied in the implementation, and changes observed.

4.    Key Evaluation Questions:

This evaluation seeks to answer the following questions (Note: the evaluator may adjust or add to these questions, in consultation with Forum for Language Initiatives, to ensure sufficient data is collected):

a)    For the Management:

• What expression of democratic and non-discrimination participation of target groups (CBOs, language communities) can be noted in terms of context analysis, formation of strategy, planning, decisions on allocations of recourses, and monitoring of the project?
• What approaches taken by the project can be regarded as innovative or as appropriate adaptations of good practice?
• What lessons learned could be used beyond the project area, in other provinces, or similar situations nationwide?
• Which particular aspects of the project, if any, could be used to influence future work in Pakistan (by FLI or others)

b)   For monitoring and evaluation of project activities and results:

• Have the project outputs and outcomes been achieved by the stated plans?
• Are project participants satisfied with the project activities? Are activities tailored to their needs?
• What activities have worked well? Why? What was the outcome?
• What activities haven’t worked well? Why?
• Are the project activities and outputs consistent with the intended objectives and outcomes?

c)    For assessing the intended and unintended outcomes of the project:

• What are the strengths and weaknesses of the approaches adopted to implement the project?
• How has the project contributed to each outcome? Who else has contributed to these changes? What is the evidence?
• What do the different actors and stakeholders perceive themselves to be the impact of the project?
• What are the major factors influencing the achievement of the project’s outcomes?
• What were the changes within the local, regional, and national-level authorities/structures in ensuring that the needs and perspectives of different Indigenous Language Communities are met during the project implementation processes about language revitalization?
• To what extent have gender and peacebuilding been considered in the project implementation?
• What are the gender implications of the identified outcomes? Do they affect women and men differently?
• How sustainable are these outcomes? Or are they one-off changes we have seen?
• Are there areas where FLI hasn’t seen any outcomes or negative outcomes? If so, why? Does FLI need to change its approach or ways of working or should FLI focus on other areas instead? and empowering language rights holders)? 
• To what degree has the project led to sustainable change (e.g. new competence, access to sustainable resources, new network partnership) for targeted language communities and reached people?
• Considering financial, institutional, and socio-cultural aspects, and regarding the original proposal, in what ways will the project have a sustainable impact in the longer term?
• Are the assumptions and external constraints originally identified in the project design still valid? What new constraints and opportunities have arisen during the project?
• To what extent did government policy support or constrain project implementation?
• What internal factors helped or hindered the project?
• What did the project do to mitigate against these external and internal constraints?

5.  Evaluation Methodology and Process:

The evaluator/firm will use the participatory evaluation methodology (Preferably Outcome Harvesting). The focus of this evaluation is to interact with various categories of target actors to establish what changes in behavior and relationships have taken place as a result of FLI interventions and what the significance of those changes are both in the short- and long-term.

It is expected that the evaluator/firm will engage with the project team, CBOs, and project beneficiaries to identify and document significant outcomes realized during the project implementation. The outcome information should be collected in the following ways:

a)       Review the project documents, including donor reports on the project outputs and outcomes, and identify preliminary outcomes.

b)      Conduct KIIs and/or FGDs with selected project participants (Indigenous Language Community members, beneficiaries, and other key stakeholders to substantiate and verify intended outcomes and to identify additional outcomes.

c)       Conduct KIIs and/or FGDs with the project team (FLI and CBOs) to expand on the identified outcomes, contextualize, and analyze their significance, and discuss how they contributed to achieving the project objectives.

d)      Collect at least two stories of change based on outcomes to be included in the final evaluation report.

The methodology should be described in more detail in the expression of interest by evaluator/firm, clearly outlining how it will be used in practice during the evaluation process.

6.  Evaluation Outputs:

The expected outputs are:

a)       Final methodology/work plan produced and shared after the briefing meeting with Forum for Language Initiatives.

b)      Draft evaluation report for FLI review.

c)       Final evaluation report, incorporating FLIs feedback. The report should be written in clear and concise English language, be of high quality to share both internally and externally, and not exceed 40-50 pages, excluding any annexes (e.g., a list of people interviewed).

d)      Conduct FGDs and KIIs, selection of participants to be determined in consultation with FLI during the briefing meeting.

e)       At least two stories of change (maximum two pages each) included in the annexes.

7.  Timeframe:

The final report should be submitted to Forum for Language Initiatives no later than July 30, 2024. The suggested activities are below, though they might be edited accordingly later. The consultant/firm will propose the timeline for the below activities:

Days Required for the following activities:

2 days for debriefing on methodology and preparation with FLI

7 days for Desk review of existing project documentation (including project proposal documents, training & advocacy materials, participants' evaluation forms, Activity reports, RBM updates, donor reports, annual reports, and relevant research papers)

10 days for Data collection/field visit to project locations to conduct FGDs and KIIs with project participants, and key stakeholders.

10 days for Writing up draft final evaluation report (including case studies) for review/feedback from FLI.

10 days for Incorporation of feedback from FLI and donor agency and production of the final evaluation report

1 day for Debrief and feedback meeting with FLI.

1.  Approaches:

a)       The evaluator/firm will work closely with the FLI MEL unit to design and undertake the evaluation, using the participatory evaluation approach.

b)      The evaluator/firm will manage the process from inception to completion, working closely with FLI; the evaluator/firm will be managed by the FLI Executive Director.

c)       The analysis, approach, and methods should adhere to FLIs conflict and gender sensitivity approaches/policies and community security model and strive for simplicity in design with practical application.

d)      The analysis, approach, and methods should be participatory, whereby project team members and participants will be involved throughout the process.

2.  Qualification, Skills, and attributes required:

The assignment will be contracted to a Consultant/Firm with experience in the following areas:

e)       Significant experience designing and leading evaluations and/or reviews (Preferably based on outcome harvesting), including a demonstrable understanding of qualitative and participatory approaches.

f)       Strong experience and communication skills to facilitate interviews (including remotely), with a range of actors.

g)      Practical experience in participatory evaluation-focused evaluation tools.

h)      Demonstrable experience in producing high-quality, credible reports in English.

i)        The firm/Consultant should know all concepts, principles, approaches, and best practices for evaluating project-based assignments.

j)        The Firm/consultant should understand institutions and project methodologies within the scope of the project to be evaluated and experience in the analysis of organizational systems and structures.

k)      Number and details of similar assignments undertaken by the firm/Consultant.

l)        Excellent attention to detail and evidence of quality outputs from previous assignments.

m)    Have a valid NTN (If Citizen of Pakistan).

9. Application Process:

FLI invites expressions of interest from evaluators/firms with the required skills and experience.

The expression of interest should comprise:

a  A cover letter outlining relevant experience, and suitability for the consultancy (maximum 1 page).

b)    An outline of the methodology proposed for the assignment (maximum 2 pages).

c)     CV/company profile, including contact details.

d)    A sample of previous work relevant to the assignment (in English).

e)     At least two references.

f)      Indicative budget (maximum 1 page) covering daily rate(s) and any related expenses.

Rates should be in USD or PKR and inclusive of VAT. The budget should be inclusive of all travel-related costs (international and local flights and in-country accommodation for international consultants/firms).

Firms/Individuals meeting the above criteria can share their proposals by June 07, 2024.

You may share the required documents through courier at the Forum for Language Initiatives, Islamabad Office