Terms of Reference for external evaluation of the project of Caritas Albania:
Community based work with marginalized children, youths and families
Project duration: 01.01.2018 – 31.12.2020 with extension until 30.04.2021
Project No.: 536-2014-001
Project donor: Caritas Germany with funds of the German Federal Ministry for Economic Cooperation and Development (BMZ)
- Project description
The project “Children and Youth“ in Albania has been implemented in three phases since January 2012 and is currently in a four month no-cost extension period, in order to enable the implementation of certain activities that could not be implemented in 2020 due to Covid-19 regulations.
Whilst in the first phase of project, the work and activities were aiming at the establishment of two daily centres for children and youth and the related services in these two centres, such as school assistance, civic education, health education, counselling, vocational education, etc., the second phase focused on laying the basis for children, youth and their families living a dignified life in their communities, involving parents and opening the centres partly in terms of community work.
In the current phase of the project, as an additional dimension advocacy and referral of individual, social and community problems and needs were taken on as a further area of work, complementing and completing the work implemented in the centres.
In all phases of the project activities centred on supporting education, school attendance and performance of the children at school, following of hygienic rules and improving communication and social skills as well as providing a first set of profession-oriented skills, due to their importance to the development of the children. Community work and advocacy to and cooperation with local authorities amplify this work and provide the necessary environment for protection and development support to the children.
During the second project phase an external mid-term evaluation took place in March 2017.
Overall and specific objectives:
Overall objective
Marginalized children, youths and their families are empowered to improve their lives as active citizens and their rights are advocated.
Objectives of the project
1: The children and youth are aware of and have improved their psycho-social health and are able to act responsible and autonomous in the Centres
2: Children, youths and their parents/caretakers have basic education, and the youths have marketable professional skills.
3: Parents are aware of parental responsibilities and are active in the community to solve common problems.
4: Governmental structures are mobilized on the importance of social services in the improvement of marginalized children, youths and families’ lives.
Target groups
Direct target group
300 children aged 5 to 11 years (60 % male and 40 % female) of all ethnic backgrounds (Roma, Egyptians, Caucasians etc.);
250 Youth aged 12 to 18 years (60 % male and 40 % female) of all ethnic backgrounds.
About 120 Parents/caretakers aged 16 to 70 years (male – female??).
About 220 individuals from the communities of Breglumi and Cerrik.
Indirect target group
360 family members
35 teachers from public schools in both areas of the project. 6 elementary schools, 2 High schools,
3 doctors and 5 nurses of both health centres in Cerrik and Breglumi.
The communities of Breglumi and Cerrik
Child Protection Units of the Municipalities.
- The aim of evaluation
Reason of the evaluation: The project is currently in its last year of implementation. The evaluation is supposed to reflect the current approach and to give recommendations how to further develop the work in the future. It is seen as a learning tool that analyses current activities and provides recommendations for improvement of the work in the future, thus also providing a basis for future acquisition of funds.
- Areas to be evaluated/focus of the evaluation.
The evaluation should cover implementation period from 1/1/2018 until 31/12/2020. For questions, with long term perspective, also the previous phases can be taken into account. The geographical areas of the evaluation are Breglumi (Tirana) and Cerrrik with the two children’s centres, as well as the head office of Caritas Albania in Tirana.
The evaluation shall look into all five DAC-Criteria, which are: effectiveness, relevance, efficiency, sustainability and impact.
With the criterion “Impact” measuring the long-term outcomes of the project, mainly previous phases should be taken into account.
Focus of the Evaluation/ Key questions:
Efficiency:
Was the project/programme implemented in the most efficient way (time, personnel resources)? Have any issues emerged, if so which ones and why?
- Cost effectiveness of the project components:
- Capacity building of the staff
- Services for beneficiaries
- Centres facilities
Relevance:
- Is the project in line with the Child Protection policies and reforms of the country?
- Is the project relevant for the needs and priorities of the target groups and the communities where the centres are located? Are feedback mechanisms for children/youth/families in place and used by them?
- Are the objectives of the programme still valid?
Effectiveness:
1) Are the provided services effective for improving the living and psychological situation of the
beneficiaries?
- Is the approach effective in improving the social and psychological health of the children/youth.
- Do the educational components give children/youth support to perform at school and to cope with their daily problems?
2) Are the joint cooperation measures with the stakeholders effective for the solving of the
beneficiaries’ problems?
- To what extent does the chosen approach contribute to activate parents and the community to act in a responsible way and to improve the social adaptation of children and youth in their communities?
3) To what extent have beneficiaries been satisfied with the programme interventions?
4) To what extent could initiatives of self-reliance be strengthened.
Sustainability:
1) Are the stakeholders ready to continue cooperation and support to the project in the future?
2) How is continuation and sustainability ensured by the project?
3) How will provision of the services be organized to the target group after the completion of the donor funding?
Impact:
- Are there long-term effects of the project interventions?
- Do project interventions make a difference to the life of the project beneficiaries (and their families)?
The evaluation should provide information on the level of impact achieved on 3 levels:
- Impact at individual level
- Impact at institutional level
- Impact at society/community level
- Procedure and methods:
Procedure:
This evaluation is a participatory evaluation, aiming at learning from successes and challenges to strengthen project strategies for the future and empowering project staff in evaluation methods (Data collection, analysis, reporting).
When all partners are in agreement with the ToR, Caritas Germany chooses the evaluator to implement the evaluation and asks Caritas Albania to agree.
The same applies for the choice of local evaluator. In this case, Caritas Albania nominates a candidate and must obtain consent from Caritas Germany.
Caritas Germany signs the contract with the evaluator selected by their side and Caritas Albania signs the contract with the local evaluator.
Then the briefing for German evaluator will be done in Germany by Caritas Germany. Further briefing for Albania and German Evaluator together will be done in Albania by Caritas Albania.
After the data collection a debriefing between the evaluators and Caritas Albania takes place.
The evaluators complete the draft evaluation report and send it to the responsible project officer at Caritas Germany who first control the date of receipt and whether improvements need to be made. The report is then sent to Caritas Albania asking to promptly provide a statement. If imperfections subsist, the project officer will discuss them with the evaluator.
Upon submission of the final version of the evaluation report, the German project officer invites the evaluator to a debriefing at the central offices of Caritas Germany. At this time the evaluation’s central aspects, results and especially its recommendations will be discussed.
And the local evaluator will present and discuss the results, findings and recommendations of the evaluation to the project staff of Caritas Albania.
Methods:
The evaluation should include but not necessarily limit to the following methods:
– interviews with representatives of the project beneficiaries (children, youth, families, caregivers);
– interviews with the project staff of Caritas Albania
– individual and/or group interviews with Stakeholders (governmental stakeholders and civil society), representatives of management of Caritas Albania
– field visits
– desk review of relevant documents (project documents, quarterly, semi-annual and annual project reports, reports on project activities, relevant national policy documents etc.);
– other methods like the most significant change, etc.
In the inception report the evaluators describe the concrete approach of the evaluation:
– the methods to be used to answer the different key questions
– the number and selection criteria of interviewees; of participants in group discussions, etc.
- Structure of the report, form of reporting and reporting language
Reporting should be presented in three distinct phases:
– An inception report shall be presented during the preparation phase; comments of the evaluation stakeholders need to be incorporated
– The draft evaluation report shall be presented after the field work; comments of the evaluation stakeholders need to be incorporated
– The final evaluation report shall be presented in agreement with the evaluation stakeholders
As for dates see chapter 7. Timetable.
Requirements for the presentation of the evaluation report:
– Maximum 20 pages for the final report, not including appendices.
– Include an executive summary, max. 4 pages.
– The report should be delivered in English and Albanian (hard copy and electronic copy)
The evaluation report should be presented according to the following structure:
- Cover sheet
- Outline / table of contents / list of abbreviations / summary / executive summary (at least one page, maximum four pages):
– Short description of the implementation organization and investigative procedures
– Aims of the evaluation and methodological approach
– Fundamental findings with reference to the main issues of the ToR
– The most important recommendations
- Evaluation report (according to the outline “areas to be evaluated” in the ToR)
- Recommendations
- Annexes
- ToR
- Timetable
- List of persons polled
- List of reference documents
- Requirements placed on the evaluator.
Criteria for Evaluators
- Personnel
– Education and work experience relevant to the task at hand
– Foreign languages: English
– Team player
– Intercultural competency
– Physical fitness
- Work experience
– Experience in development cooperation
– Work experience in evaluation of programs / projects
III. Specialist
General evaluation-specific competencies
– Theoretical and methodological basics of evaluation (basic terms, concepts, etc.)
– Planning and implementation of evaluations (incl. budgeting, contractual agreements)
– Professional standards
– Specific methods and concepts of evaluation (and quality control)
Methodical competencies
– Methodology and methods of empirical social research (quantitative and qualitative procedures, simple and complex procedures of data analysis, data processing)
– Methods of business management (e.g., accounting)
– Evaluation methods
– Methods of project management (methods of time, cost, and implementation planning)
Context-related competencies
– Expert knowledge in the respective environment
– Knowledge regarding general conditions/structures in the environment (especially policies which have direct impact on evaluations in this area)
– Specific models used in the environment, procedures, and methods of evaluation
– Interdisciplinary, general knowledge in the areas of organization, communication, etc.
For application evaluators should provide
– CV
– Copies of the previous evaluations
– References
– Motivation letter
– Evaluation proposal (including methods, activities, timeframe, and budget)
NOTE: the documents of application should be sent on hard copy in a closed envelope.
- Timetable:
27.01.2021 – announcement for the selection of the local evaluator
7.02.2021 – application deadline for local evaluator
10.02.2021 – interviews for selection of the local evaluator
15.02. – 25.02.2021 – submission of joint inception report
26.02. – 26.03.2021 – data collection, implementation of the evaluation
27.03. – 7.04.2021 – submission of the draft report
9.04.2021 – presentation of results and recommendations of the draft report to Caritas Albania project staff by local evaluator.
20.4- 23 .04.2021 – submission of the final report
- Responsibility
Ms. Enkelejda QAMA, Program Manager of Caritas Albania will be the contact person for this evaluation/review.
Contact details:
Mob: ++ 355 69 70 41 303
Tel: ++ 355 (04) 22 300 88
Fax: ++ 355 (04) 22 343 27
Email: e.qama@caritasalbania.org
Address: Rr. “Don Bosko“ Nr 4, Tirana, Albania
Telephone: 00355 4 223 00 88
E-mail: caritasalbania@caritasalbania.org